Can Semantic Labels Assist Self-Supervised Visual Representation Learning?

Recently, contrastive learning has largely advanced the progress of unsupervised visual representation learning. Pre-trained on ImageNet, some self-supervised algorithms reported higher transfer performance compared to fully-supervised methods, seeming deliver message that human labels hardly contribute transferrable features. In this paper, we defend usefulness semantic but point out and methods are pursuing different kinds To alleviate issue, present a new algorithm named Supervised Contrastive Adjustment in Neighborhood (SCAN) maximally prevents guidance from damaging appearance feature embedding. series downstream tasks, SCAN achieves superior previous sometimes gain is significant. More importantly, our study reveals useful assisting opening direction for community.